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Abstract 



This paper reports the results from an investigation of a method for 
systematic development of a ptocedure for assessing the administrator and 
instructional leader roles it. ementation of an innovative school program. 
The purpose of :he study was co design and field-test a feasible and useful 
approach to examining the degree of opemcionalization of certain administrative 
roles and responsibilities that have been identified as critical for performance 
by school principals and other designated instructional leaders in order to 
provide adequate implementation support. Instrument design procedures are 
described and results from the study are summarised based on data from 19 
teachers, five principals, and four education specialists from five elementary 
schools where an innovative educational program known as the Adaptive Learning 
Environments Model (ALEM) was implemented. The implications for further 
research in the development of procedures for assessing the degree of program 
implementation and in the use of implementation assessment data also are 
discussed. 



Evaluating Administrative ^jmponents of Program 
Implementation in the Adaptive Learning 
Environments Model 

The measurement of program implementation variables and effects in eval- 
uation research in education has received considerable attention during the past 
few years. Educational program developers, researchers and evaluators recognize 
that utilization of the traditional "treatment/yield" paradigm within classical 
experimental designs is a necessary, but insufficient condition to adequately 
understand how and why innovative programs work. Tht use of classical pretest 
and postest control group experimental designs, while useful from what Popham 
(1975) refers to as the "conclusion-oriented" research perspective, leaves much 
to be desired from the "decision-oriented" perspective of the educational 
evaluator. Demonstrating that a particular program treatment had a statistical- 
ly significant effect is certainly important. However, this information may not 
be especially useful to adopters/adaptors of innovative programs who usually 
entertain more extended questions such as "What makes the program work?... What 
elements of the program need to be implemented (and at what levels) to make the 
program work?... What are the critical features of the program what should be 
observed if it is indeed in place and operative?", and so on. 

These questions center on the measurement of elements of program implemen- 
tation. They suggest that evaluators of educational innovations should collect 
such information to better understand the how's and why's of program effects. 
The focus of the measurement and evaluation of program implementation variables 
is program description, development, modification, improvement, adoption, staff 
development, and th>» like. The importance of assessing program implementation 
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variables and the decision focus for evaluating program implementation have been 

recently elucidated by Wang and Ellett (1982): 

Innovative educational interventions that effectively improve 
student learning can have a substantial impact on existing schooling 
practices — if the innovations are used. An educational program with 
the most impressive evidence of effectiveness can have no effect on 
classroom processes and student learning unless teachers, administra- 
tors, and relevant school personnel accept the ideas and procedures 
and decide to use them. The extent to which any innovative program is 
used, however, depends on a host of factors. A critically important 
factor is evidence cf effectiveness, that is, the program's outcomes 
and the conditions under which it was implemented to produce those 
outcomes. The availability of information on the potential impact of 
the program on the school's day-to-day operations is crucial. Infor- 
mation is needed on who does what to produce what mediating effects, 
what processes lead to certain intended outcomes, and what contexts 
and conditions are critical for program implementation. Potential 
program adopters and users must be able to determine not only expected 
program performance but also the nature of specific links between the 
implementation of each aspect of the program and its intended out- 
comes. Program validation , therefore, should provide such information 
so that meritorious educational inno\ations can be described in ways 
that facilitate teachers 1 and administrators' decision making (p. 
35-36). 

The need for developing instrumentation and assessment methodologies to 
measure the degree of program implementation is well documented in the recent 
literature on program evaluation (e.g., Ellett, 1981; Fullan and Pomfret, 1977; 
Hall and Loucks, 1977; House, Glass, McLean & Walker, 1978; Rutman, 1980; Wang 
and Ellett, 1982). However, methodological advances and measurement tech- 
nologies have been rather slow to develop. This seems partially due to the 
expense and time involved in conducting direct observation and interview studies 
(the preferred methodology) and difficulties in developing assessment procedures 
that have sufficient validity and reliability to measure program implementation 
elements and characteristics. The study described in this paper was designed to 
further understanding of the administrator and instructional leader roles of 
school principals and others in supporting the implementation of innovative 
school programs. Specifically, the study was conducted to test a system of 
assessment and instrumentation for monitoring and evaluating the degree of 
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implementation of both roles. The particular innovative program with which the 
implementation assessment system was field-tested was the Adaptive Learning 
Environments Model (ALEM) . Tn the first section of the paper, design and 
rlassroom operation of the ALEM are briefly described to provide background 
information. In the second section of the paper, the development of the 
implementation assessment system for the administrator and instructional leader 
roles is discussed. Then, the procedures, instrumentation, and results from the 
study are presented. Finally, the implications for future research and far 
further development of effective program implementation monitoring and 
assessment systems are discussed. 

The Adaptive Learning Environments Model 
The Adaptive Learning Environments Model (ALEM) is an innovative program in 
early childhood education that has been under development and study at the 
Learning Research and Development Center, University of Pittsburgh, for over a 
decade. The central goal of the ALEM is to provide learning environments that 
are responsive to the characteristics and learning needs of individual students 
(Wang, 1980). The program's design systematically integrates aspects of 
individualized prescriptive instruction that facilitate basic skills master> 
(Bloom, 1976; Glaser, 1977; Rosenshine, 1979) with aspects of informal edu ion 
that foster self-responsibility (Johnson, Maruyama, Johnson, Nelson, & Skon, 
1981; Marshall, 1981; Peterson, 1979). The ALEM consists of 12 critical program 
dimensions. Nine are related to the process of providing adaptive instruction. 
They are Creating and Maintaining Instructional Materials, Developing Student 
Self-Responsibility, Diagnostic Testing, Instructing, Interactive Teaching, 
Monitoring and Diagnosing, Motivating, Prescribing, and Record Keeping. Three 
of the dimensions — Arranging Space and Facilities, Establishing and Communi- 
cating Rules and Procedures, and Managing Aides — are related to providing 
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support for implementation of adaptive instruction in the classroom. In 
addition to the 12 dimensions, implementation of the ALEM is supported by a 
delivery system that is made up of four major components: an ongoing, 
data-based staff development approach; instructional teaming; multi-age 
grouping; and family involvement. 

The expected outcomes of the ALEM for students include effective use of 
school time, personal motivation to spend the time required to master basic 
academic skills, end development of competence and responsibility for managing 
learning and elements of the learning environment. Teacherb in ALEM classrooms 
are expected to be able to spend significantly more time managing learning than 
managing behavior. (For more information on the design, implementation, and 
effects of the ALEM, see Wang, Gennari, & Waxman, 1985, and Wang & Walberg, 
1983). 

An underlying assumption of the ALEM's design is that, for most schools, 
program implementation requires some fundamental changes in curriculum mate- 
rials, instructional philosophy and procedures, organizational and staff support 
systems, teaching and learning processes, and teachers' and students' roles in 
the learning environment. Thus, the measurement of such program implementation 
variables in past studies of the effectiveness of the ALEM has been a constant 
concern. The Implementation Assessment Battery for Adaptive Instruction (Wang, 
1980; Wang, Catalano, & Gromoll, 1983) is used by school personnel on a regular 
bat's to collect information for staff development purposes and to monitor the 
overall degree of implementation of the ALEM in their classrooms. Implementa- 
tion data also are collected three times during the school year for program 
evaluation purposes. A total of 106 performance indicators, which are grouped 
in the Battery's six data collection forms, have been identified to assess the 
presence and absence of the critical program design dimensions of the ALEM 
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(Wang, Nojan, Strom, & Walberg, 1984). 

The measurement of program implementation variables in past studies of the 
effectiveness of the ALEM has always been a concern. A recent study by Strom 
and Wang (1982) reported the results of validating the ALEM degree of program 
implementation instruments for classroom level variables in 138 classrooms. 
Classroom processes measures, student learning outcomes and staff development 
criteria were examined for these classrooms in relation to scores on the ALEM 
program implementation measures. The results suggested a moderate to high 
association between these variables, supporting the criterion-related validity 
of the program implementation measures used iri evaluating implementation of the 
ALEM. Additionally, generalizability analyses supported the reliability of the 
ALEM implementation measures in differentiating ALEM components and characteris- 
tics and generalizing over observers and assessment occasions. 

School principals and education specialists play important supportive and 
administrative roles in the implementation of the ALEM. They are involved in 
initial program startup, program monitoring and maintenance and providing 
ongoing staff development for ALEM teachers. They are considered critical to 
the success of the level of implementation of the ALEM by the manner in which 
they provide supervision, supportive and consult ive services, and leadership for 
classroom teachers. Indeed, recent research reviews of the literature on 
effective schools (Mackenzie, 1983) show that the quality of these support and 
leadership roles is an integral part of school effectiveness and productivity. 

Purpose 

The purpose of the present study was to develop and pilot test an 

assessment procedure to examine the roles, responsibilities, and functions of 

school principals and education specialists in implementation of the ALEM. To 

date, there are no systematic measures of the performance of these roles as 
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they relate to overall program implementation. The "grapevine structure" among 
teachers implementing the ALEM, the experiences and observations of ALEM devel- 
opers and consultants, and the philosophy and operating assumptions of the 
program all suggested the need to develop and pilot-test en assessment system 
for measuring implementation of the administrative and instructional leadership 
components of the ALEM. 

Methodology and Procedures 

Subjects 

Subjects for the study were school principals and education specialists 
implementing the ALEM in selected classrooms in five elementary schools in a 
large urban school s>stem during the 1982-83 school year. The school principals 
and the ALEM education specialists were considered the primary administrative 
support personnel for teachers in the ALEM classrooms. 
Instrumentation 

Two kinds of instruments were developed for this initial pilot study: 
1) interview frameworks to be administered by an "outside" interviewer; and 2) a 
survey to be administered to all ALEM teachers. The interview frameworks 
assessed the degree to which school principals and education specialists imple- 
mented the roles, duties, and responsibilities of administrators and 
instructional leaders under the ALEM. One interview framework focused on the 
school principal, and the other focused on the education specialist. 
Information on teachers' assessments of the roles/duties carried out by 
principals and education specialists was obtained via the ALEM Program 
Implementation Survey for Teachers (hereafter referred to as the Teacher 
Survey) . 

During the initial stages of the study, a process model for developing and 
revising a system for assessing the administrative and instructional leadership 



7 



components of the ALEM was designed. The general sequence of elements in thz 
model is presented in Figure 1. The primary focus of Steps I through VI was to 
establish content validity of the instrument items and to develop adequate 
operational definitions for administrative roles, duties, and responsibilities 
of the ALEM principal and education specialist in terms of observable behaviors. 
The ALEM program developer and staff development coordinator (the most highly 
experienced persons in the ALEM) worked closely with an outside consultant in 
the development/design of the instruments. Development of final items and a 
framework for the interviews and the Teacher Survey required approximately four 
days. Steps VI through X in Figure 1 depict the sequence of data collection, 
analysis, and revision of the pilot instruments. 

Completion of Step IV yielded a large matrix of job duties and responsibil- 
ities for the ALEM principal and education specialist that could be 
operationalized in terms of observable behaviors and cross-classified by appro- 
priate assessment data sources (e.g., interviews, direct observation, 
information obtained from parents, teachers, and students). Almost all the 
behaviors identified with these procedures were considered capable of being 
assessed by either an outside interviewer/observer or ALEM classroom teachers. 
The decision was made to collect data for each of the behaviors by using a 
trained outside interviewer (one with knowledge of the ALEM's components and 
goals, but with no prior interaction with ALEM school site personnel in the 
pilot schools) and the Teacher Survey. 

Each of the performance indicators of the 12 critical program design 
features of the ALEM was defined further with separate sets of scoreable de- 
scriptors for the roles of school principal and education specialist. Two kinds 
of scoreable descriptors were developed: 1) hierarchical, sequential 
descriptors scored on a 1- to 5-point scale; and 2) sets of four essential 
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Step I_: 

Establish Role 
Definitions — - 
Broad Categories 
and Subelements 



Step X : 

Revise/Adjust ^— 
Pilot Instruments 



Stej) II: 

Define Sub-elements 

in Terms of Job 

Duties 



Step III : 

Establish Behavioral 
Referents for 



Specific Functions 



Step IX: 

Data Scoring 
and Analysis 



Step VIII : 



Collect Pilot <- 
Test Data 



Step IV; 

Cross-Classify 
-> Behavioral Referents 
with Appropriate 
Data Sources 



Step V: 

Develop Methodologies 
for Measuring 
Behaviors — 
(Descriptors) 



S tep VII : 



Develop Data 
Collection Plan 



Step VI: 

Finalize Pilot 
Instruments and Data 
Collection Methods 



Figure 1. Process model for developing, piloting, and revising the instruments for assessing the implementation of the 
ALEM by principals and education specialists. 
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behaviors which served to operationalize a particular performance indicator 
(also scored on a 1- to 5-point scale). For the first kind of descriptor, a 
score of 2 was considered an "adequate" level of implementation of ALEM program 
behaviors. A score of 1 was viewed as an "inadequate" level of implementation 
and a score of 5 was considered a "superior" level of implementation of ALEM 
pro&ram behaviors. Scores of 2 or A were considered somewhat less than adequate 
and somewhat more than adequate, respectively. 

Two kinds of descriptors were required in developing the scoring methodolo- 
gy for the implementation measures since some indicators could not be clarified 
or scaled for scoring with the hierarchial, sequential methodology. APPENDIX A 
contains sample sections of the interview frameworks for the school principal 
and education specialist to illustrate the essential desciiptors methodology and 
the hierarchical, sequential descriptor methodology for selected performance 
indicators. 

The final assessment framework for principals consisted of three job roles 
(^Instructional Leadership, Communications, find Program/Management/Support) 
defined by 15 essential performance indicators (each scored on a 1 to 5 scale 
with a set of scoreable descriptors). The fin'l assessment framework for 
education specialists consisted of eight key job roles (?re-service Training, 
Curriculum Planning/Updating, Monitoring Program, Staff Development, Instruc- 
tional Support, Family /Community Involvement, Documentation of Program Effects, 
and Communications) defined by 19 essential performance indicators (each scored 
on a 1- to 5-point scale with a set of scoreable descriptors). A list of all 
the performance indicators identified for the rolet; cf the principal and the 
education specialist under the ALEM and comprising tie final assessment 
frameworks is found in APPENDIX B. The complete interview l-dineworks display 
each of the role definitions, 6ubelements, performance indicators, and scale of 
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scoreable descriptors that were developed pnd judged as / appropriate measures of 
implementation of the administrative and instructional leadership components of 
the ALEM (Ellett & Wang, 198A). 

The Teacher Survey consisted of all 34 performance indicators and associ- 
ated descriptors for the principal and education specialist roles. Specific 
questions relative to each performance indicator solicited teachers' judgments 
about the functioning of their school principal and education specialist within 
the ALEM. These questions were designed to parallel the kinds of questions for 
each performance indicator that were included in the interview frameworks. 
Sample pages from the Teacher Survey lorm are presented in APPENDIX C. 
Data Collection 

All data for the study were collected during the late spring of 1983 over a 
five-day period. Two methods were employed. The first required that the 
outside interviewer collect interview data using the assessment frameworks 
developed for the education specialist and school principal. Each school 
principal and education specialist was interviewed individually. In order to 
collect additional ALEM implementation dita and to corroborate information 
obtained from the principal and education specialist, two ALEM teachers were 
randomly elected within each for interviews. These teachers were also 

interviewed individually. All interviewees were informed of the nature of the 
pilot study and the fact that their data were contributed confidentially. Using 
this interview design, four interviews were held in each ALEM school — one each 
with the school principal and the education specialist and one with each of the 
two randomly selected teachers. Usinf only one interviewer served to reduce 
between-interviswer variability in the data collection procedure. 

All interviews were undertaken using the interview frameworks as e system- 
atic guide. Open-ended questions were asked which i zused specific responses 
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for scoring each of the scoreable descriptors. Probing questions were asked 
when further clarification and additional explanation/evidence was needed to 
check a particular descriptor. The specific content of each descriptor state- 
ment was used as a basis for crediting an item. Sometimes this procedure 
required an observable demonstration on the part of the principal or education 
specialist ; sometimes it required strong* verbal corroboration from the teacher 
being interviewed. For example, performance indicator #2 in the principal 
interview framework suggests that the principal should be able to identify and 
explain the 12 critical dimensions of '.he ALEM. When interviewing the princi- 
pal, the interviewer would ask hor many of the critical dimensions could be 
identified and would request an actual (though brief) explanation of each 
dimension identified. The set of descriptors would then be scored based on the 
resulting data provided by the principal. When asking a teacher about this 
particular item, the interviewer would accept the teacher* s best judgment as to 
how many critical dimensions could be identified by the school principal and how 
many of these could be explained. 

Principals and education specialists were asked to produce as much visible 
evidence/documentation as possible to corroborate their degree of implementation 
of each performance indicator. Teachers, on the other hand, were more typically 
asked about their knowledge of a principal's or education specialist's 
demonstration of the various descriptors and they were asked to provide examples 
of specific evidence. 

For each descriptor, where sufficient evidence/documentation existed, a 
check mark was made on the interview form in the appropriate space In the 
columns to the right of each statement (see samples in APPENDIX A). When data 
were collected during the principal or education specialist interviews, check 
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marks were made lr. the column labeled "External". When the first teacher was 
interviewed, data were recorded in the column labeled "T/l". Data for the 
second teacher interview were recorded in the column labeled "T/2". The "T/3" 
column was not used in this study since only two teachers in each school were 
interviewed. 

The Teacher Survey was administered to all ALEM teachers in each of the 
five participating schools. Teachers were asked to complete the survey within 
three days and to return their completed anonymous surveys for processing in a 
stamped envelope provided. Nineteen useable surveys were received from the 28 
ALEM teachers (68%) in the five elementary schools. 
Data Scoring and Analysis 

All raw data from the irterview frameworks and Teacher Surveys were key- 
punched as either one's or zero's for the sets of descriptors defining the 
various performance indicators. This allowed each indicator to be scored for 
each interviewee or survey respondent on a 1- to 5-pcint s^ale. If no descrip- 
tors were credited for an individual interviewee or respondent, the indicator 
score was equal to 1 . If one descriptor was credited, the indicator score was 
equal to 2. If two descriptor* were credited, the indicator score was equal to 
3 : and so on. 

The sequential, hierarchical sets of descriptor* were keypunched ranging 
from 1 to 5 based on the appropriate space checked to the right of each descrip- 
tor (scale point) . 

Three kinds of analyses were performed on the data: 1) Descriptive statis- 
tics for each performance indicator and instrument subscalc (ALEM "role" aggre- 
gation) for the total sample and for each school, for the principal and the 
education specialist separately; 2) intercorre] ations between instrument sub- 
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scales for the responses to the Teacher Survey; and 3) indices (percentages) of 
agreement between the external interviewer's implementation scores on the 
interview frameworks for the principals and education specialists and the scores 
from the teachers interviewed. These analyses allowed for an examination of 
variation in implementation levels, interrelationships between implementation 
scores across teachers for ALEM principals and education specialists, and the 
utility/potential reliability of the interview procedure as a means of 
collecting/corroborating program implementation data within the ALEM. Two 
supplemental analyses were also completed. 

Results 

Results from the study are summarized under four major headings. First, 
responses to the ALEM Program Implementation Survey for Teachers (Teacher 
Survey) are presented in terms of (a) a summary of teachers 1 perceptions of the 
roles played by the principals and education specialists in program 
implementation, and (b) the findings from e correlational analysis of the survey 
results for the 19 responding teachers. Next, the results from the principal 
and education specialist interviews are summarized. Findings from the ana^sis 
of interrater agreement between the ratings on the principal/education 
specialist interviews and the teacher survey are presented under the third 
heading. Finally, the results from supplemental analyses are discussed. 
Descriptive Statistics and Comparisons: Teacher Survey 

Table 1 presents a summary of means and standard deviation for the first 
15 performance indicators derived from the 19 responses to the Teacher Survey. 
(See APPENDIX B for statements corresponding to the numbering of performance 
indicators). These data are for the role of the school principal as viewed by 



14 



the ALEM teachers. The indicator means ranged from a high of 4.21 (#1 ~ prin- 
cipal's ability to identify and explain the three basic components of the ALEM) 
to a low of 2.71 — principal's functioning in the area or communicating 

with parents and the community). Nine of the 14 indicator means reported in 
Table 1 exceeded 3.00, which was originally established as the score point 
representing an "adequate" degree of implementation of ALEM roles and respon- 
sibilities. None of the indicator means approached the highest score of 5.00, 
and only two exceeded 4.00. This finding suggests that teachers viewed their 
principals as implementing their ALEM responsibilities at an "adequate" or 
"above adequate" level, but that the degree of total implementation was not 
nearly as hfgh as it couid be. 

Tab] a 2 displays descriptive statistics for each performance indicator in 
the Teacher Survey for the education specialist's ro^e. As shown in Table 2, 
the performance indicator means ranged from a low of 2.69 (#11 - education 
specialists' functioning relative to the design of staff development plans) to a 
high of 4.36 (//9 - education specialists' interactions with principals to 
discuss ALEM program implementation data). All performance indicator means 
except one (//ll) exceeded the scale midpoint (3.00 « "adequate"). Seven indica- 
tor means exceeded 4.0, and 13 of the 19 indicator means exceeded 3.50. 

An analysis of snbscale scores for the Teacher Survey (administrator's role 
and education specialist's role) also was completed for each of the five sample 
schools to examine between- and within-school variability in degrees of imple- 
mentation of the ALEM. These results show considerable variability in the 
degree to which teachers viewed their principals and education specialists as 
performing their administrative and instructional leadership duties and 
responsibilities under the ALEM. These results are presented in Tables 3 and 4. 
It should be noted that the score? are not directly comparable between subscales 
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TABLE 1 

Summary of Descriptive Statistics for Performance 
Indicators on the Teacher Survey: 
School Principal's Role 
(n ■ 19 Teachers in 5 Elementary Schools) 



Performance Indicator X S .D. 

1. 4.21 0.85 

2. 3.74 1.41 

3. 2.83 1.34 

4. 3.43 1.70 

5. 3.57 1.28 

6. 4.17 1.03 

7. 2.76 0.97 

8. 3.35 1.00 

9. 2.71 0.99 

10. b b 

11. 2.83 0.92 

12. 2.82 1.51 

13. 3.78 1.26 

14. 3.72 1.45 

15. 3.06 1.30 



Note : Numbering of performance indicators corresponds with numbering in 
APPENDIX B. 

k This indicator is not scored on a I to 5 scale. 
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TABLE 2 

Summary of Descriptive ftatistics for Performance 
Indicators on the Teacher Survey: 
Education Specialist's Role 
(n « 19 Teachers in 5 Elementary Schools) 



Performance 
Indicator 






X 


S.D. 


1. 


3.64 


1.65 


2. 


3.63 


1.26 


3. 


4.19 


1 .22 


4. 


3.13 


1.30 


5. 


4.11 


0.90 


6. 


3.78 


1.31 


7. 


4.00 


1.22 


8. 


3.89 


1.23 


9. 


4,36 


0.92 


10. 


3.40 


1.30 


11. 


2.69 


1.03 


12. 


3.18 


0.95 


13. 


3.53 


0.64 


14. 


4.26 


0.81 


15. 


3.59 


1.00 


16. 


3.39 


1.04 


17. 


3.08 


0.79 


18. 


4.17 


1.29 


19. 


4.11 


1.15 



Note: Numbering of performance indicators corresponds with numbering in 
APPENDIX B. 
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since the number of items on which they are computed varies from one subscale to 
the next* 

Subscale Intracorrelations; Teacher Survey 

Subscale scores for the Teacher Survey were developed for the set of 15 
performance indicators reflecting the role of the school principal and the 1^ 
performance indicators reflecting the role of the education specialist. Pearson 
product-moment correlation coefficients were computed using 6ubscale scores, 
with teachers as the unit of analysis, to examine the direction and strength of 
relationship between subscales. 

Table 5 displays intracorrelations between the three subscales of the 
Teacher Survey: Principal's icole. All correlations were statistically signifi- 
cant (p .01), positive, and ranged from a low of r*.60 (Program Super- 
vision/Instructional Leadership) to a high of r».75 (Instructional Leader- 
ship/Program Management). These correlations are considered moderately high and 
demonstrate from 36% to 56% common variation among subscale scores. Considered 
collectively, these results suggest that principals who were viewed positively 
in their functioning under the ALEM by teachers in one program area generally 
were viewed positively in other program areas. 

Table 6 displays the results of subscale intercorrelations for the Teacher 
Survey responses for the education specialist's role. Correlations varied from 
a high of r*.86 (Communications/Instructional Support) to a low of r«.04 (Family 
Community Involvement/Documentation of Program Effects). With two exceptions, 
all correlations between 6ub6cales were positive. The correlations ranged from 
demonstrating rather strong relationships between subscales to little or no 
relationship at all. Because of the rather small sample size, some of the more 
moderate correlations (e.g., approaching .40) were not statistically significant 
(p .05). These findings show generally moderate to rather high intracorre- 
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TABLE 3 



Summary of Descriptive Statistics from the 
Teacher Survey by School and Subscale: 

School Principal's Role 
(n = 19 Teachers in 5 Elementary Schools) 



Subscalt 



School 


Instructional 


Leadership 


Program 


Suoertlslon 


Conaanlcatlons 


Prograa Organization/ 
Manaffement/SuDDort 




Mean 


(S.D.) 


Mean 


(S.D.) 


Mean 


(S.D.) 


Mean 


(S.D.) 


A 


9 67 


(6.66) 


7.60 


(3.64) 


6.33 


(3.06) 


12.33 


(8.39) 


B 


12. 76 


(3.20) 


8.00 


(1 00) 


9.00 


(2.16) 


18.26 


(6.02) 


C 


16.33 


(2.62) 


7. CO 


(4.36) 


9.00 


(6.00) 


16.33 


(6.11) 


D 


11 00 


(6.20) 


3.33 


(1.63) 


7.33 


(6.61) 


12.67 


(6 13) 


E 


13 80 


(3.19) 


6.20 


(311) 


6.80 


(164) 


16.60 


(6.31) 
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TABLE A 



Summary of Descriptive Statistics from the 
Teacher Survey by School and Subscale: 

Education Specialist's Role 
(n ■ 19 Teachers in 5 Elementary Schools) 



School 







A 




B 






C 






D 


i 




E 






Subscala 


Moan 


(S.D.) 


Mean 


(S.D.) 


Mean 


(S.D.) 


Mean 


(S.D.) 


He an 


(S 


D ) 


Training 


7 


00 


(4.38) 


14 


26 


(6.24) 


16. 


00 


(6.93) 


13 


33 


(4.73) 


10 


60 


(6 


11) 


Curriculum 
Planning/Updating 


0 


00 


(2.66) 


12 


60 


(3.70) 


0 


67 


(4.73) 


8 


67 


(3.21) 


10 


00 


(6 


16) 


Prograa 

Monitoring 


7 


.00 


(2.00) 


8 


00 


(4.00) 


6 


67 


(4.16) 


6 


00 


(2.83) 


6 


20 


(2 


.77) 


Staff 

Development 


8 


07 


(2.6)2 


12 


76 


(2.63) 


16 


00 


(6.29) 


6 


60 


(3.64) 


9 


80 


(3 


.83) 


Instructional 
Support 


11 


.87 


(0.68) 


11 


76 


(IBP) 


10 


67 


(4.61) 


9 


.67 


(3.21) 


9 


80 


(3 


96) 


Family/Community 
Involvement 


3 


.00 


(141) 


3 


.00 


(1.00) 


4 


00 


(0.00) 


3 


.00 


(0 . 00) 


3 


00 


(0 


.00) 


Documentation of 
Prograa Effects 


4 


.33 


(116) 


4 


.26 


(160) 


4 


.00 


(173) 


3 


.67 


(1.63) 


4 


26 


(1 


.60) 


Communications 


4 


.33 


(0.68) 


4 


.60 


(100) 


4 


.00 


(173) 


3 


.00 


(100) 


4 


.20 


(1 


.30) 
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TABLE 5 




Summary of Intercorrelations Between Subscales 
of the Teacher Survey: School Principal's Role 
(n * 19 Teachers in 5 Elementary Schools) 




Subscalt 


Sub6cale 


instructional Leadtrsblp Prograa Suptrfislon CowunlcatlOM 


Predate Suptrfislon 
CoM^ni cations 
Prograa Hanagtstnt 


.GO** 
.06** 

.6*** 74*** •••••• 


**p < 01 
***p< .0001 
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TABLE h 



Summary of Intercorrelations Between Subscales 
of the Teacher Survey: Education 
Specialist's Role 
(n ■ 19 Teachers in 5 Elementary Schools) 

Subscale 



Subscale 


Training 


Curriculum 

Planning/ 

Updating 


Program 
Monitoring 


StL*f 

Development 


Instructional 
Support 


Family/ 

Community 

Involvement 


Documentation of 
Program Effects 


Curriculum 

Planning/ 

Updating 


.66** 














Program 

Monitoring 


.36 


.48 












Staff 

Development 


.66* 


.69** 


.62** 










Instructional 
Support 


.60* 


.83*** 


.62** 


.67** 








Family/ 

Community 

Involvement 


.67* 


-.16 


37 


.46 


.19 






Documentation of 
Program Effects 

Communication* 


.39 
.30 


73*** 
.83*** 


.62* 
.67** 


.30 
.66** 


.7B*** 
.86*** 


.04 

-.06 


.79*** 



•p<.05 
•*p<.01 
***p<.001 
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lations between subscales of the Teache" Survey for the education specialist's 
role. As with the findings for the principal's role, they suggest that educa- 
tion specialists who were viewed by teachers as positively performing their role 
in one program area, generally were viewed as positively performing in many 
other program areas. 

Descriptive Statistics: Implementation Interviews 
for Principals and Education Specialists 

Individual item and subscale scores were computed for the implementation 
interviews for the principal and education specialist from each school. Scores 
were derived for each performance indicator by adding the scores for the two 
teachers and either the principal or the education specialist interviewed in 
each school. The maximum possible raw score for a given indicator on either 
interview Instrument was 15.00 (3 interviews X 5). 

For the school principals, total ALEM implementation scores between schools 
ranged considerably — from 113 to 181 with a maximum possible total score of 
225 (15 indir- -rs X 3 interviewees X 5 - 225). When converted to percentages 
of the maximum possible total score, these implementation scores ranged from 
50.2% to 80. 4%. For education specialists, the total ALEM implementation scores 
between schools ranged from 228 (80% of che maximum possible score of 285) to 
275 (96.5% of the maximum possible total score of 285). 

When considered collectively, these results suggest higher overall degrees 
of implementation in the sample schools for the educat'.on specialists than for 
the school principals. More variation in scores also was evident among 
principals than among education specialists. 
Interrater Agreement Analyses; Implementation Interviews 

Interrater agreement analyses of the interview data were undertaken to 
check on the extent of agreement between the external interviewer's recording of 
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information from the ALEM principals and education specialists and the informa- 
tion received from the fvo randomly selected teachers within each school. An 
index of agreement was constructed using the scale midpoint for each indicator 
(3 * "adequate" level of Implementation) as a recommended miniuum implementation 
level. Therefore, the agreement index of concern was the number (and percent- 
age) of scores for a particular indicator that met or exceeded a score of 3.00. 
This index was used because of the "pilot" nature of the study and the instru- 
ments, the difficulty in obtaining high indices of exact agreement in new 
interview settings, and the belief that corroborating implementation at or 
exceeding the minimum level of implemencation was the most important concern. 

Scores tabulated for each indicator were converted to agreement scores. 
Only raw scores which were 3.00 or greater were considered in computing these 
agreement scores. If only one of three 6cores met or exceeded the minimum level 
(3.00), the performance indicator vas scored as 0.00. If two of the three 
scores met or exceeded the minimum level, the performance indicator was scored 
as 67%. If all three of the scores met or exceeded the minimum level, the 
performance indicator was scored as 100%. Data for each principal and education 
specialist were scored in this manner and a percentage of agreement at or 
exceeding the minimum level for each performance indicator was computed across 
all five ALEM schools. 

Agreement percentages for the performance indicators ranged from 0% (prin- 
cipal's understanding of key role responsibilities of program staff) to 100% 
(principal's maintenance of a master schedule which reflects ALEM requirements, 
and principal's performance in allocating and supplying instructional mate- 
rials). Agreement percentages (means) for each subscale (Instructional Leader- 
ship, Communications, Program Organization/Management/Support) were 62%, 67%, 
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and 70%, respectively. The total agreement percentage (grand mean percentage) 
for the entire interview procedure for principals was 66%. 

For the education specialist interview instrument, perfornance indicator 
agreement percentages ranged from 100% (for 8 of the 19 indicators) to 42% the 
education specialist's involvement in planning pre-service training for ALEM 
staff and volunteers). Mean percentages of agreement for the various subscales 
were: Pre-Service Training (73%); Curriculum Planning/Updating (86%); Monitor- 
ing Program (96%); Staff Development (91%); Instructional Support (90%); Fami- 
ly/Community Involvement (71%); Documentation of Program Effects (100%); and 
Communications (100%). The grand mean percentage of agreement for the complete 
interview instrument for education specialists was 86%. These results suggest 
that agreement percentages "at or exceeding the minimum perfoTmance level" were 
greater for the education specialist in the sample schools than for the school 
principals. 
Supplemental Analyses 

Two supplemental analyses of the ALEM program implementation data were 
undertaken to: 1) explore relationships between teachers 1 views of principals' 
end educational specialists' performance of their ALEM-related tasks; and 
2) compare schools in a gross ranking procedure across the four implementation 
measures utilized in the study. 

Table 7 displays the intercorrelations between the Teacher Survey subscale 
scores for the principal and education specialist performance indicators. All 
of the correlations were positive in direction and ranged from rather low 
(r«.18) to moderately high (r«.73). For the table total, 59% of the possible 
intercorrelations were statistically significant (p .05), with six other corre- 
lations approaching significance. Considered as a whole, the results in Table 7 
suggest that the degree to which school principals were viewed by teachers as 
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TABLE 7 



Summary of lntercorrelations Between Subscales 
of the Teacher Survey for the Principal's Role 
and the Education Specialist's Role 
(n = 19 teachers in 5 ALEM schools) 



Education Spnclal Subscalns 



Principal 
Subscalts 



Instructional 
Luadursblp 

Program 
Supervision 



Family/ Documentation 
Curriculum Program Staff Instructional Community of Program 

Training Planning Monitoring Dnrnlopmnrt Support Involveannt Effects Communications 



46* 

.40 
63*» 



.43 

.43 
.62* 



Communications 

Program Organization/ 
Hanageaent/Support .86*** .71*** 



.18 

.72*** 
.61* 

.47* 



.30 

.67* 
.43 

.47* 



.46* 

88** 

.63** 

.73*** 



.19 

.43 

.33 

21 



.36 

.43 
.46* 

.34 



.46* 

.67** 
60* 

.64** 



* p< 06 
** p< 01 
««« p<001 
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implementing their ALEM-related job responsibilities was positively and signifi- 
cantly related to the degree to which the education specialists were viewed as 
performing their roles in the ALEM, 

In order to examine the consistency in rank of the five sample schools 
across all four implementation measures, a gross index of rank was computed for 
each measure for each school. Each school was rank ordered in terms of the 
grand mean scores for the principal and education specialist interview proce- 
dures and total scores for the forms of the Teacher Survey. The results showed 
that no strong "winners' 1 appeared across all four measures. However, of eight 
of the possible first- or second-place rankings across the four measures, either 
of two schools appeared five times, with four of these rankings resulting frou 
the Teacher Survey data. These results, while certainly not definitive, can be 
interpreted as showing the beginnings of an implementation level trend in the 
ALEM's administrative and insf— tional leadership components in these five 
sample schools. 

Discussion, Implications, and Recommendations 
The process of developing and pilot testing a set of assessment procedures 
to measure implementation of administrative components of the ALEM yielded 
several interesting and important results. To date, implementation levels in 
the ALEM have been assessed by focusing on teacher and student behaviors and 
classroom level variables. The implementation measures reported here targeted 
the job duties, roles and responsibilities of the two key administrative and 
instructional leadership personnel in the ALEM system... the school principal and 
the education specialist. Both of these individuals are assumed to be important 
administrative and support elements in the degree and success of ALEM 
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implementation. Recent research on school improvement provides much evidence 
that implementation of administrative and instructional leadership structures, 
are important elements of school productivity (Block, 1983). 

Before this study, no serious effort had been put forth to design and pilot 
a system for measuring levels of ALEM implementation with the school principal 
and the educational specialist as the objects of measurement. The job analysis 
procedure utilized (See process model in Figure 1) proved to be quite useful in 
developing and operationalizing a system for collecting data relative to crit- 
ical ALEM performance indicators for these two key educators. A similar method- 
ology has been successfully utilized in developing measures to assess 
performance competencies of school administrators (Ellett & Payne, 1978) and 
beginning teachers (Capie, Anderson, Johnson, Ellett, 1980). To the authors 
knowledge, the methodology reported here has not been previously utilized to 
develop measures of program implementation. The procedures not only served to 
identify critical performance indicators for the roles of the principal and the 
education specialist in the ALEM, but they also were used to place the 
performance indicators in a scoreable framework in both interview and survey 
formats. 

Analyses of the interview framework data across the five pilot schools 
indicated that these procedures are, for the most part, sensitive to variation 
in levels of ALEM implementation. As with all interview procedures, an element 
of subjectivity exists in deciding whether a particular verbal response or other 
documentation produced by an interviewee should be "counted" as evidence of 
program implementation. If the interview procedures are used on a large scale 
basis, a training program for interviewers to standardize questions and 
scoreable evidence for the job descriptors would be required. Focused training 
of interviewers and perhaps the development of a catalogue of evidence for each 



35 



28 

performance indicator and scoreable descriptor can serve to control possible 
sources of unreliability when using interview methodologies like those used in 
this study. 

The agreement data reported for the interview frameworks is important for 
several reasons. First, the findings suggest that agreement rates were general- 
ly higher for the education specialists then for the school principals. Several 
interpretations of this finding can be made. However, the most useful interpre- 
tation may have to do with the "visibility" of the education specialist in the 
ALEM school environment. Teachers may be able to more reliably report on 
aspects of the specialist's functioning than the school principal's functioning 
due to the specialist's more intensive involvement (and thus visibility) in the 
ALEM program. The interrater agreement rates for the interview data established 
in this pilot study provide some support for this interpretation. 

Secondly, the procedure used to establish agreement percentages is impor- 
tant when considering the notion of "minimum" levels of implementation in the 
ALEM and similar programs (setting program standards). Traditional approaches 
to establishing interrater agreement percentages for observers or interviewers 
are usually estimates of "exact" agreement. Exact agreement procedures may not 
always be a useful index for evaluation procedures when program implementation 
is measured. What seems important from the evaluation perspective, is the 
extent to which agreement exists at or above established minimum performance 
standards. The reliability concern in this context, is program implementation 
decision making. This seems an important concern because three sources of data 
(principal, teacher #1 and teachar #2) could all judge a performance indicator 
at or above a level of 3.00 (minimum level) with 0% exact agreement (e.g., 
principal - 3; teacher # 1 - A; teacher #2-5); but 100% agreement relative to 
the established minimum performance level (3.00). 
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With thi? view in mind, the results of this pilot study suggest that, for 
the mosc part, minimum program implementation levels were being met. This was 
particularly the case for functioning of the ALEM education specialists. In 
establishing standards for the ALEM and similar programs, what seems important 
is whether the resultant program implementation data show consistency at or 
exceeding the recommended program performance standards. Disagreement above or 
below the established performance standards does not seem to be a natter of 
concern in evaluating levels of implementation relative to a standard, though it 
may well be important from the "conclusion-oriented 11 research perspective 
(Popham, 1975). More refined studies of these issues seem needed. 

Intracorrelations between subscales of the school principal and education 
specialist Teacher Survey data l ^gested that the quality of performance in one 
job area was positively associated with many other job areas. This finding 
suggests that principals and education specialists who are viewed by ALEM 
teachers as high performers in some program areas are also viewed as high 
performers in other program areas. Additionally, intercorrelations between 
subscales of the two instruments suggest that high performance of education 
specialists (from teachers' perspectives) is associated with high performance of 
school principals. It would be interesting in future studies to examine the 
relationship between ALEM classroom level implementation measures with those 
used in this study. Linkages between implementation of administrative and 
instructional leadership support roles and implementation of classroom behaviors 
of teachers and students in the ALEM could be established. 

Future studies are needed or. Luc measures developed and piloted in this 
investigation. The instruments and procedures need to be field tested in a 
larger context to see if the findings reported here can be replicated. It might 
prove useful to establish the concurrent validity of the implementation measures 
and procedures used by comparing "new" ALEM school site to ALEM school sites 
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that have been implementing the model for a number of years. 

If the measures used in this study could be administered to a much larger 
sample of ALEM school sites, more could be learned about the relationship 
between and usefulness of teacher surveys and interviews as program 
implementation methodologies. This is a trying task, since school means should 
be used as the units of analysis in these studies and a reasonable number o r 
schools would have to be included to maka the findings statistically robust. 

Finally, some attention should be given in future research to examine 
relationships between levels of implementation with the measures piloted in this 
study and learner outcomes. Ideally, the evidence from such criterion-related 
validity studies would show that schools profiled by high implementation of ALEM 
administrative roles/responsibilities are providing optimum learning environ- 
ments for children and are especially productive in terms of learner outcomes. 
Tying program implementation variables to learner outcomes in innovative 
educational programs like the ALEM should be an important, ongoing concern in 
future evaluation research studies in education. 
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APPENDIX A 



Sample Performance Indicators 
and Scoreable Descriptors for 
the Principal and Education 
Specialist Implementation 
Measures in the ALEM 
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HIERARCHICAL, SEQUENTIAL METHODOLOGY FOR DESCRIPTORS 



Performance Indicator 2: Explain tne data base used to desip.n/develop staff 

development plans for teachers. 

Scale of Descriptors 

ALEM SCHOOL PRINCIPAL 
External T/l T/2 T/3 



1. No systematic data base or printout is evident. 

2. A computer printout is evident (available) but no knowledge 
of content is demonstrated. 

3. A computer printout is evident (available) and specific 
content can be briefly explained. 

4. Ways in which data are used to develop specific staff 
development plans can be explained in detail. 

5. In addition to #4 above, the manner in which supplemental 
information is used to design/develop staff development 
priorities can be explained. 

*If the Implementation Assessment Battery for. Adaptive Instruction is 
used to evaluate performance of teachers, a score of "1" should be fciven. 

Evidence/Ration ,le: [ _ 
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SET OF ESSENTIAL BEHAVIORS METHODOLOGY FOR DESCRIPTORS 
Instructional Lendership Role 

B. Staff Development 

Performance Indicator 1: Plan/coordinate staff development activities for teachers. 



Descriptors 



ALEM SCHOOL PRINCIPAL 
External T/l T/2 T/3 



a) Individualized ALEM staff development plans can be 
identified. 

b) Components of individual staff development plans can be explained 
or elaborated in terms of function/usc/nature of activities. 

c) Most recent, required staff development plans are 
available for esch ALEM teacher. 

d) Staff development responsibilities of the education 
specialist can be explained in detail. 



Kvidence/Rationale: 
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HIERARCHICAL 4 SEQUENTIAL METHODOLOGY FOR DESCRIPTORS 
I Pre-service Training Role 

A. Planning 

Performance Indicator 1: Plans pre-service 'raining for ALFM program 

staff and volunteers. 

Scale of Descriptors 

ALEM Education Specialist 
External T/l T/2 T/3 



1. No pre-service planning has occurred. 

2. Pre-service planning has been done; but no written 
plans are available. 

3. Written pre-service plans are evident; but are 
incomplete. 

4. Written pre-service plans specify objectives, 
activities/materials, nnd responsible persons. 

5. In addition to #4, written plans specify one or more 
ways to monitor trainee progress toward objectives. 

Evidence /Rationale: . 
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SET OF ESSENTIAL BEHAVIORS METHODOLOGY FOR DESCRIPTORS 



B. Implementing 

Performance Indicator 1: The content of pre-servlce training centers 

on essential elements of the ALEM. 



Descriptors 



ALEM Education Specialist 
External T/l T/2 JOJL 



a) Pre-servlce training Includes an explanation of tl 
rationale and design for the A1.IM. 

b) Pre-servlce training includes an explanation of tl 
major components of the ALEM. 

c) Pre-servlce training Includes an explanation of a 
12 critical dinensions of the ALEK. 

d) Pre-servlce trplnin* results in an individualized 
staff development pi in for each teacher. 



Evidence/Rationale: 
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APPENDIX B 



List of Essential Performance 
Indicators for the Principal 
and Education Specialist 
Roles under the ALEM 
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ESSENTIAL PERFORMANCE INDICATORS FOR THE THREE ROLES OF 
THE SCHOOL PRINCIPAL UNDER THE ALEM 

I. Instructional Leadership 

Performance Indicators 

1. Identify and explain the three basic components of the ALEM. 

2. Identify and explain the 12 critical dimensions of the A^EM. 

3. Plans/coordinates staff development activities for teachers. 

4. Explain the data base used to design/develop staff development 
plans for teachers. 

5. Monitors program implementation through classroom observation. 

6. Periodically meets with the education specialist to discuss 
program implementation. 

II. Communications 

7. Establishes a network or structure for communicating with ALEM 
staff. 

8. Communications are effective. 

9. Establishes a network or structure for communicating about the 
ALEM to parents , the community* and others. 

III. Program Organization/Management /Support 

10. Maintains a master schedule for utaff which reflects ALEM re- 
quirements. 

11. Implements staff development program(s). 

12. Understands key role responsibilities of ALEM program staff. 

13. Allocates and supplies instructional materials. 

14. Allocates and supplies human resources. 

15. Implements a process for the identification and placement of 
special needs children. 
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ESSENTIAL PERFORMANCE INDICATORS FOR THE EIGHT ROLES 
OF THE EDUCATION SPECIALIST UNDER THE ALEM 

I. Pre-Service Training 

Performance Indicators 

1. Plans pre-service training for ALEM program staff and volunteers. 

2. The content of pre-service training centers on essential elements 
of the ALEM. 

3. In-service training includes essential activities. 

4. Assesses effectiveness of pre-service training. 

II. Curriculum Planning/Updating 

5. Analyzes basic skills curriculum to determine degree of individ- 
ualization • 

6. Coordinates the individualization of the curriculum. 

7. Facilitates the individualization of the curriculum by identify- 
ing supplemental materials and learning activities. 

III. Monitoring Program 

8. Monitors program implementation through classroom observation. 

9. Periodically meets with the principal to discuss program imple- 
mentation. 

IV. Staff Development 

10. Uses Implementation Assessment Batter for Adaptive Instruction as 
a basis for planning. 

11. Designs individualized staff development plans. 

12. Conducts in-service training activities. 

13. Assesses effectiveness of in-service activities. 
V. Instructional Support 

14. Diagnoses learning needs of individual students. 

15. Develops instructional plans with teachers. 

16. Provides/coordinates instructional resources and services which 
meet students* needs. 
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VI . Family/Communi ty Involvement 

17. Plaus/participates in family and connuiniuy invci.veme.i- activ- 
ities. 

VII. Documentation of P r ogram Effects 

18. Coordinates and implements data collection activities. 

VIII. Communications 

19. H ?int i structure 'Tor communicating with ALEM ««*aff. 
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Sample Pages from the Teacher Survey 
for Role of the Principal and the 
Education Specialist 
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PART I 

*. 

Please answer each item by marking the appropriate space (s). If uncertain , 
leave the iten: blank. 



1. If my principal was asked to identify and explain the three basic com- 
ponents of the ALD1, he/she could (check only one ) 

not directly identify any ALD1 component. 

directly identify one component. 

directly identify two components and briefly explain them. 

directly identify all three ALD1 components and briefly explain then. 

directly identify all three ALEM components and explain each in 

detail. 



2, If my principal was asked to identify and explain the twelve critical 
dimensions of the ALDi, he/she could ( check only one ) 

identify fewer than three critical dimensions. 

identify and briefly explain three or more critical dimensions. 

identify and briefly -explain six or more critical dimensions. 

identify and briefly explain nine or more critical dimensions. 

identify and briefly explain twelve ot more critical dimensions. 

3. In planning/coordinating staff development activities for teachers, 
my school principal (c heck only one ) 

has knowledge of individual staff development plans. 



can explain individual staff development plans in terms of 

function/use and nature of activities. 

maintains the most recent, required staff development plans for 

each teacher. 

can explain staff development responsibilities of the education 

specialist in detail. 

In designing staff development plans for teachers my principal ( check only one ) 

does not maintain any systematic data base or printout. 
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__ maintains a computer printout but does not understand its content. 

maintains a computer printout and can explain its content. 

can explain in detail ways in which data are used to develop 

specific staff development plans. 

can explain ways in which data are used to develop specific staff 

development plans and the manner in which staff development pri- 
orities are established. 

5. In monitoring ALE>> program implementation in the classroom my principal 
(check all that appl y) 

systematically monitors the learning envi'onment (learning centers, 

physical arrangements of the classroom, etc.). 

systematically monitors instructional procedures/activities. 

assesses pupil progress toward learning objectives (spot checks 

classroom records or profile charts)* 

discusses results of classroom observations with teachers and/or 

paraprof essionals • 

6. When my principal meets with the educational specialist to discuss pro- 
gram implementation ( check all that apply ) 

formal and informal implementation data are discussed. 



specific strengths and weaknesses at the classroom level are 

identified. 

results of major stf.ff development activities are discussed. 

plans for continuing and new staff development activities are made. 

7. In establishing a network or structure for communicating with ALEM staff 
my principal ( cht-k all that apply ) 

maintains in written form rules/procedures/policies for communica- 
tions. 

maintains a file of written communications (memos, letters, documents, 

etc. )« 

_ periodically holds informal meetings with staff. 

holds regularly scheduled meetings with instructional staff. 
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PART II 



Please answer each it en by marking the appropriate space (s). If uncertain , 
leave the iter, blank. 



In planning pre-service training for ALDi progran staff and volunteers, 
the education specialist in this school ( check onlv one) , 

does not do pre-service planning. 

does pre-service planning, but does not develop written plans. 

develops written pre-service plans, but these are somewhat incomplete. 



develops written pre-service plans which specify objectives, activities/ 

materials and responsible persons. 

develops written pre-service plans which specify objectives, activities'' 

materials, responsible persons and one or more ways to monitor trainee 
progress toward objectives. 

2. In conducting pre-servic training the education specialist in this 
school ( check all that apply ) 

explains the rationale and design of the ALE1 program. 

explains the major components of the ALE! program. 

explains all twelve critical dimensions of the ALD! program. 

develops an individualized staff development program for each teache* > 



During in-service training the education specialist in thib school, 
( check all that apply ) 

uses large group discussion about rssant^al elements of the AL 

program. 

explains essential elements of the ALEM program in detail. 

applies elements of the ALD1 program to teacher's classrooms. 

includes a "closing session 11 to synthesize information about 

implementation of the ALEM program. 
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